Coinbase Chat MCP

Your API is not an MCP | DEMFP786 https://www.youtube.com/watch?v=eeOANluSqAE All right. Um, nice to see you everyone. Thank you for joining. Um, my name is David. I work at a company called Neon. Uh, we're a serverless Postgres provider. So, the people want Postgress, we give them Postgress. And today, I'm going to be talking about MCP servers. Uh in specific I'm going to be talking about um how not to build MCP servers um and then how to actually go about it um in the right way. So, if you're unfamiliar with MCP, um it's a new protocol that was developed by Anthropic, which is um a way for LLMs to be able to interface with real world apps and services. So essentially it's a protocol an open protocol that standardizes how applications provide context to LLMs um and how LMS can use real world products. It's very um new. It's about uh six seven months old but it's getting tremendous adoption. So if you saw the keynote yesterday from Sodka uh the Windows co-pilot will become an NCP client soon. Just today in the morning at uh Google code they announced or Google IO rather they announced that Gemini will support MCP as well. OpenAI already supports MCP and Claude of course has supported MCP from the beginning. What does that mean? It means that if you're building an app or a service and you want LLMs to use your app, you need an MCP server. Um so this has led to a lot of companies building MCP servers in a rush um in order to make their services available to LLMs like Cloud and OpenAI's Chat GPT. So let's break it down. What makes up an MCP server? Uh MCP servers have tools, resources, and prompts. These are the three main concepts of an MCP uh server. And these are the things that an MCP server exposes to the underlying LLM, which is the MCP client. The most important of these by far are the tools. Tools, you can think of them as actions. These are things that the LM might want to perform. An example would be if you're building an e-commerce website to buy an item on your website. Um, in our case, we're a Postgress provider. We give people Postgress. Um so an example of a tool would be to create a Postgress database. Um and this would be a request that comes in from the LLM. The resources and prompts are not as um interesting. So I won't I won't spend any time on them today. Um if the concept of an MCP server still hasn't um permeated, it's it can be like a tricky thing to to really understand. I have a very quick demo here of Neon's MCP server um in production. So I have a video here of um a cursor user uh asking cursor to create an application using neon. And what cursor will do and I know you can't see the video um but what cursor will do is it'll interface with our MCP server in order to request a Postgres database and it will then use said Postgress database um to create an application. So if you look at the prompt there it says build me a to-do list app use neon postgres and use neon o and what cursor will do is it'll use our MCP server to provision a postgres database with neon o um and in just a few minutes the application will be fully functional and have a real database a real physical postgress database thanks to our MCP server. So MCP is awesome. I think we've established that. Um the problem now is creating an MCP server is a lot of work. Our MCP server, which is actually open source, um is around 500 lines of code, so it's not that not that large, but it's still, you know, a significant amount of work to put it together. You have to write tests. You have to think of it. And so um a lot of companies have decided to just autogenerate their MCP server and in fact a lot of services recently came out that help you do this. So how does that work? Every API has an open API spec an open API schema which describes all the endpoints the inputs and outputs to all of those endpoints. in theory um well not just in theory is I mean um quite simply one could take an open AI open API schema and just autogenerate an MCP server Siri got in the way uh um you can just take an open API schema and autogenerate an MCP server it's very easy it'll take less than a minute and then uh you have an MCP server now the problem with this is that it isn't quite right. So while it it is very easy and like I said there's a bunch of services like stainless speak easy mintify and there's even a few more um it isn't the right thing to do. Let's talk about why. The first problem is that APIs tend to be very extensive. Um, in the case of Neon's API, we have around 75 to 100 different endpoints. I'm not showing them all on this slide, but we have a lot of endpoints. And LLMs are really, really terrible at choosing from a long list of tools. So, if you give them too much choice, they won't really know what to do. They'll get very confused. Even though context windows have been increasing and you hear about a million tokens, 5 million tokens, unlimited token context window, that doesn't mean that an LLM performs just as well with a large context than it does with a smaller context. In fact, the opposite is true. LMS perform much better with a reduced context size. So, if you give them too many tools, they won't know what to do. So if you autogenerate an MCP server, you're going you're going to get your all of your API endpoints as MCP tools and LLMs will not perform well against your service. So you have to make a choice here. Then the second problem is that API descriptions in your existing API are probably not that well written for LLMs. This is sometimes more true than others, but for the most part, in my experience, um, API endpoints are written for humans who can Google things and can reason about things, but LLMs need you to be a lot more direct with them. And also, LLMs need examples much more than humans do. So in our case for our MCP server, our uh tool descriptions are actually written in XML and we write them in this very um organized manner and we try to give the LLM as much context as possible about each individual tool and when to use it which is something that you probably are not doing in your APIs today. So, in a way, writing for an LLM is different than writing for a human, which is why you want to think about how to write the descriptions of all of your MCP tools uh for LLMs. And then what you should also think about doing is writing tests for this. So, we have emails, which are basically uh tests in the AI world. We have eval to make sure that LLMs are calling the right tool for the right job. And we run these eval 100, a thousand, 10,000 times because obviously LLMs are not deterministic. And we make sure that the tools that we're exposing to the LLM um have good descriptions and we iterate on these descriptions as our uh eval. The third problem is that most APIs out there today are designed for low-level resource management and automation. The reason most of uh businesses have an API are for developers to go and use those APIs for automation. But this is not what LLMs need from an API. LMS need tasks. They need actions and tools. LM are much more humanlike in that sense. And so an LLM doesn't really care about like low-level um resource creation. It cares about achieving a specific goal. And so when you design an MCP server, when you design the list of tools, you need to design that with um with that in mind. And and this is not what you do with an API. And that's part of the reason why Enthropic decided to create MCP in the first place is because Open API schemas out there today are not really prepared for this kind of um design. And then finally, if you just take your API and expose it as an MCP, you're missing out on the potential to create many more interesting things. Um, and I'll walk you through an example from our MCP server. So one of the things that LLMs need to do when they're building an app is do database migrations. That's true about humans. That's true about LLMs. And so we have to expose this in our MCP server so that LLMs using Neon for Postgress can do database migrations. The naive approach is to expose a single tool called run SQL and then expect LLMs to call that tool to do database migrations. Um so if we have a tool called run SQL which we actually do in our MCB server an LLM can go and use it and pass through any um alter table statements that it wants and just do database migrations like that. But it's a lot more interesting to use the opportunity that you're developing an MCP server to think about um doing more than just the basic. So when we created our MCP server, we decided to expose purpose-built MCP tools for database migrations. So we have a prepare database migration tool and we have a complete database migration tool. And LLMs um are eager to use these tools over the run SQL tool. We actually encourage LMS to use these tools if they if they're wanting to to do database migrations on their Neon database. Um, and the way it works is the first tool sort of stages the database migration on a temporary neon branch. And then once that's done, we actually respond back to the LLM and say, "Hey, the database migration you're trying to run is now staged on this branch. Please go and test the the migration before you commit it." And then we actually teach the LM how to commit the migration. So you can see in the end we say call the finished database migration to complete this um job and then the LM can call the complete database migration when it decides that it's um ready to go for their main database branch. This kind of thing we wouldn't really expose it in our API. It doesn't really make sense to have such a complex multi-step workflow in our REST API. But it is totally the kind of thing that makes a lot of sense for LLMs, especially because LLMs are not that good at at SQL. And so we kind of want to help them test their SQL before they apply it on their main database branch. So if you're building an MCP server today, if you haven't built your company's MCP server yet, what should you do? Should you write it from scratch or should you autogenerate it using one of the tools that I mentioned before? I would say definitely don't autogenerate it. I don't I really don't think that's the way to go. But a hybrid solution where you start by autogenerating an MCP server and then cut it down can make a lot of sense. I haven't tried it myself, but in principle, you should be able to autogenerate an MCP server from your open API schema and then cut down as many tools as possible. The worst thing you can give an LM is too much choice. So remove any tools that you don't think are essential for an LLM to consume and then evaluate the descriptions for all of your tools and then think about interesting tools that you might want to expose to an LLM but you maybe don't want to have in your API and then write your emails. Uh I that that would be a whole another talk. But if you're building an MCP server, you should have emails um and you should have tests to ensure that LLMs can use your MCP server correctly. That's it for me today. Um I just want to say like please reach out. I'm I'm an MCP nerd. I'm a database nerd. I'm happy to talk about uh remote MCPS MCP servers. Uh we've been running our MCP server in production. uh for more than a month now. Uh if you want to talk about tests or emails for MCP servers, uh happy to n about about that as well. Authentication for MCP or just anything database related. Um I'm really into databases. That's it for me today. Thank you all very much for your time. [Music]